United States Patent m 

Zetts 



USO054O4458A 

[ii] Patent Numben 
[45] Date of Patent: 



5,404,458 
Apr. 4, 1995 



[54] RECOGNIZING THE CESSATION OF 

MOTION OF A POINTING DEVICE ON A 
DISPLAY BY COMPARING A GROUP OF 
SIGNALS TO AN ANCHOR POINT 

[75] Inventor: John ML Zetts, Falls Church, Va. 

[73] Assignee: International Business Machines 
Corporation, Armonk, N.Y. 

[21] Appl. No.: 201,147 

[22] Filed: Feb. 24, 1994 

Related ILS. Application Data 

[63] Continuation of Ser. No. 774,488, Oct 10, 1991, aban- 
doned. 

[51] IntCL* G06F 15/02 

[52] ILS. CI 395/275; 364/927.6; 

364/927.64; 364/929.12; 364/DIG. 2; 345/173 

[58] Field of Search 395/140, 141, 142, 143, 

395/147, 162; 345/145, 156, 157, 158, 159, 160, 
161, 162, 163, 164, 167, 173, 179, 180; 
364/927.5, 927.61, 929.12, 929.3 

[56] References Cited 

U.S. PATENT DOCUMENTS 



4,334,266 
4,409,836 
4,587,630 
4,760,386 
4,886,941 
4,899,138 
4,903,012 
5,025,411 
5,027,109 
5,157,384 
5,179,368 



6/1982 
10/1983 
5/1986 
7/1988 
12/1989 
2/1990 
2/1990 
6/1991 
6/1991 
10/1992 
1/1993 



Mirbach 

Comroe et al. ... 
Straton et al. .... 

Heath et al 

Davis et al 

Araki et al 

Ohuchi 

Tallman et al. 
Donovan et al. 
Greanias et al. . 
Lippihcott 



.... 364/167 

...... 73/504 

.... 364/900 

.... 340/709 

.... 340/710 

.... 340/712 

.... 340/709 

.... 364/900 

.... 340/706 

.... 340/706 

.... 340/708 



5,187,467 
5,189,403 



2/1993 Myers 

2/1993 Franz et al. 



340/706 
340/706 



FOREIGN PATENT DOCUMENTS 

63-311426 4/1987 Japan . 
62-080724 12/1988 Japan . 

Primary Examiner— Thomas C. Lee 
Assistant Examiner— Moustafa Mohamed Meky 
Attorney, Agent, or Firm— George E. Grosser; Joseph C. 
Redmond, Jr. 



[57] 



ABSTRACT 



A method and apparatus for efficiently distinguishing 
between different types of input signals simulated by a 
pointing device coupled to a multi-tasking computer 
system. The pointing device may be a stylus, finger or 
other device that moves across the surface of a touch 
screen or the like to generate positional information. 
Depending on the response of a delay timer, the motion 
of the pointing device is recognized by software appli- 
cation programs as input information either from a 
mouse or from a gesture or a handwriting input mode. 
If motion cessation across the screen is detected with a 
predetermined time-delay period, the system accepts 
the input information in a mouse-emulating mode. If 
motion is detected within the predetermined time-delay 
period, the timer is reset. Thus, the system overhead 
associated with managing the timer can be reduced 
because the timer is periodically reset and need not be 
reset after each movement of the pointing device as in 
the prior art. The periodic resetting can be based on 
intervals of n points generated by a moving pointing 
device, or n real-time units relative to a given point 
generated by the pointing device. 



16 Claims, 11 Drawing Sheets 
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finger, can be categorized either as a mouse input signal 

RECOGNIZING THE CESSATION OF MOTION OF or as a gesture or handwriting input signal. Where such 

A POINTING DEVICE ON A DISPLAY BY input signals are intended to emulate the behavior of a 

COMPARING A GROUP OF SIGNALS TO AN mouse and represent commands, such as mouse button 

ANCHOR POINT 5 down and mouse button up, the stylus or finger is re- 
spectively touched down and lifted off the surface. 

This application is a continuation of U.S. patent appli- Where the input device allows the user to use a stylus 

cation Ser. No. 07/774,488, filed Oct 10, 1991, now and touch sensor to simulate pen and paper to create a 

abandoned. handwriting input signal, alphanumeric characters can 

TECHNICAL FIELD 10 ^ e entered into an appropriate application program. 

. Where the input signal is part of a gesture, a series of 

This invention generally relates to mput devices for a such mput signals resembling a geometric figure, such 

data processing system and, more particularly, a as a circle, a right-hand or a left-hand arrow, are indica- 

method and apparatus for reducing system overhead tive ofan action to be taken by the computer system. As 

associated with distmguishmg between input signals 15 ^ ^ w ofinput signals ^ emulated by the 

generated by different types of pointing devices. The me of a s ^ lus or ^ ^ a touch m ut devi it would 

invention has particular application in a muki-tastang ^ advantag eous to design a stylus- or finger-based 

computer system m which a single task is dedicated to operatillg stem such ^ ^ three ^ of f nput sig . 

controlling a timer used to distmguish between the ^ ma * b * ^^^y entered adhere on the 

mput signals. 20 ^ pUy 

BACKGROUND ART In order to utilize all three types of input signals, they 

The use of a touch input device disposed over the J?* bt m £S° g fif ^ ^^^B system 

viewing surface of a computer displa^tTprovide a P"* 1 *? 1 pf differentiating between the types of 

"user friendly" means for the control lof a data process- 25 mpU * *f f 15 by Ummg - f ! he *f > ™ ***** 

ing system is well known in the art U.S. Pat No contact between a pointing device and the touch sensor, 

5,025,411 to Tallman et al. discloses a typical input moves *? P° f mtm S device to a desired position and 

device in the form of a touch screen used to control the f t0 P s motlon f ? r a Predetermined time period without 

operation of a digital oscilloscope. U.S. Pat No. losmg contact between the device and the touch sensor 

4,587,630 to Straton et al. discloses a programmable 30 (hereinafter referred to as "lift-off), the operating sys- 

touch screen emulating a keyboard input device. U.S. tem wU1 recognize an input signal at the desired position 

Pat. No. 4,903,012 to Ohuchi discloses a touch screen 35 a mouse command. For example, if the user stops 

for calibrating the coordinates of a computer system. moving the pointing device at a given position for 200 

Such devices are designed to allow an unsophisticated milliseconds, a mouse command at the given position is 

user to perform desired tasks on a computer system 35 recognized. If, on the other hand, the user does not stop 

without extensive training. Human factor studies have ** m y & yca Position for the specified time delay period 

shown that an input device which allows the user to ^d instead lifts off the touch sensor, the input signals 

input data directly on the computer display achieves 216 selected as candidates for character or gesture rec- 

greatest immediacy and accuracy between man and ognition instead of mouse commands, 

machine. 40 To detect the cessation of motion at a desired posi- 

In current graphical user interfaces developed to aid tion » tne operating system repeatedly resets a delay 
man-machine interaction, there are many items, such as software timer each time a position point identified by 
menu selections, icons or windows, which a user can 311 m P ut signal generated by the pointing device is suffi- 
most easily select by using a mouse or a finger. U.S. Pat ciently offset from a previous position point Repeat- 
No. 4,886,941 to Davis et al. and Japanese Publications 45 edty resetting the timer in this manner creates a large 
Nos. 62-80724 and 63-311426 describe prior art systems amount of processing overhead when the cessation 
employing mouse-pointing devices as input devices for detection is required in a multi-tasking system, 
data processing systems. Some advanced softward ap- A multi-tasking computer system creates the illusion 
plications allow a user to input complex control signals °f concurrently running a plurality of computer pro- 
with a finger. U.S. Pat No. 4,899,138 to Araki et at, for 50 cesses, also known as jobs, tasks or threads. To create 
example, discloses a touch panel for providing control this illusion, an operating system switches a single pro- 
signals in accordance with the manner in which a finger cessor between multiple threads. A timer thread is dedi- 
touches the panel within a predetermined period of time cated to mana g ing the software timer. The timer thread 
(i.e., making a gesture). cannot reset the timer until the processor saves all com- 

A stylus has proven to be more effective in inputting 55 putations made while executing the current thread and 

information on a touch sensitive input device because of switches back to the timer thread. As a result, a large 

its greater precision. It is more convenient to utilize a amount of processing time is required to repeatedly 

touch input system which allows both a stylus and a reset the above-described timer, 

finger to operate as input devices. One such system is cttmmap v ™? run ixn/cxmnM 

described in commonly assigned U.S. Pat No. 4,686,332 60 SUMMARY OF THE INVENTION 

to Greanias et al., entitled "Combined Finger Touch It is an object of this invention to reduce the over- 

and Stylus Detection System for Use on the Viewing head associated with a delay timer which is reset each 

Surface of a Visual Display Device," and which is time an input signal, representing a point on a workpad, 

hereby expressly incorporated herein by reference. generated by a pointing device is sufficiently spaced 

In a touch input device (e.g., a touch workpad) for a 65 from a previous input signal, 

data processing system, where a touch sensor is dis- It is another object of this invention to maintain the 

posed over a viewing surface of a display, input signals accuracy of measurement of a period of cessation of a 

generated from a pointing device, such as a stylus or pointing device's motion while a timer is reset on every 
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nth position point and the pointing device has stopped FIG. 2 is an architectural diagram of the combined 

sometime between timer resets. finger touch and stylus detection system. 

It is a further object of the invention to more effi- FIG. 3 is a representation of a window displayed on 

ciently identify an input signal generated by a pointing the touch workpad in FIG. 1 and illustrating a stroke 

device in a multi-tasking computer system that dedi- 5 made by a pointing device that has been recognized as 

cates a task or thread to managing a timer used to distin- a mouse command. 

guish between types of input signals generated by a FIG. 4 is a flow diagram illustrating the operation of 

pointing device. the present invention to distinguish between stroke 

It is also an object of the invention to reduce in a data input signals intended to produce mouse commands, 

processing system the overhead associated with a delay 10 and stroke input signals intended to produce gesture or 

timer by resetting the timer periodically when a point is handwriting events. 

generated from a pointing device within a given period FIG. 5A and 5B are flow diagrams illustrating the 

of time starting from the time an initial point is gener- operation of the invention in reducing system overhead 

ate d. associated with resetting a delay timer to determine 

These and other objects and features of the invention 15 whether a cessation of motion has occurred, 

are accomplished by a method and apparatus which FIGS. 6, 7, and 8 are representations of a window 

manage in the following manner a delay timer in an displayed on the touch workpad of FIG. 1 and depict- 

operating system of a computer receiving input from a ing an inked stroke intended to produce a mouse com- 

pointing device. At touch-down time, the identity of the mand. 

pointing device is established, and the rate at which 20 _ ^ 0 ^ TWT , T ^ T 
points (represented by the input signals) are generated DETAILED DESCRIPTION OF THE 
by the pointing device is determined. A timer reset INVENTION 
divide rate is calculated such that the delay timer is FIG. 1 shows a touch workpad substantially similar 
reset on every nth point generated, even though motion to that described in co-pending application No. 
of the pointing device may have been detected on the 25 07/351,227 to Arbeitman, et al., entitled "Flat Touch 
current point. Points are collected as the pointing de- Screen Workpad for a Data Processing System," filed 
vice is moved across a touch-sensitive computer dis- May 15, 1989, which is hereby expressly incorporated 
play. In the alternative, points are collected from a herein by reference. The workpad 10 comprises a hous- 
pointing device that causes a cursor to move across a ing 12 having a rectangular recessed window 14 which 
display screen without requiring the screen to be 30 surrounds the edges of a rectangular touch overlay 16. 
touched by the pointing device, e.g., mouse, joystick, The touch overlay 16 is transparent and is disposed on 
etc. In response to a determination that motion has a liquid crystal display (LCD) 18. The overlay 16 con- 
occurred within the previous n points generated from sists of a laminate structure including several plastic 
the pointing device, the timer is reset with a predeter- substrate layers laminated together by means of adhe- 
mined time-delay period. If motion has occurred within 35 sive layers. The overlay 16 also includes a first plurality 
the previous n points, but has ceased during the genera- of transparent X conductors 16A disposed in the verti- 
tion of the last several points, the timer is reset with an cal direction and a second plurality of transparent Y 
additional amount of time attributable to such last sev- conductors 16B disposed in the horizontal direction, 
eral points in order to maintain the accuracy of the Several of the conductors in both the vertical and hori- 
timing. If it is determined that the motion of the point- 40 zontal directions are positioned beyond the recessed 
ing device ceased for the predetermined time-delay window 14 to allow more accurate determination of the 
period, e.g., 200 milliseconds, an action is performed by location of the stylus 20 or a finger on or near the over- 
the computer system. In the preferred embodiment, the lay 16 at the edges of the display window 14. 
action is a mouse command, such as a mouse button The stylus 20 is connected to the touch workpad via 
down or mouse button up, that is generated at the point 45 cable 22. The stylus 20 acts as an antenna to pick up the 
where motion ceased. If the pointing device does not signals radiated by the overlay 16, and provides much 
stop at any position for the predetermined time-delay greater resolution than can be provided by a finger 
period, but instead lifts off, the set of points generated touch. Also on the bezel of the housing are four button 
by the pointing device is sent to a character- or gesture- switches 24-27 which can be used to change the mode 
recognition unit as candidates for character or gesture 50 in which the data from the workpad 10 is received, 
recognition. Workpad cable 28 connects the workpad 10 to the com- 
In another embodiment of the invention a given point puter with which the user is communicating. The work- 
is generated from the pointing device and marked as a pad cable 28 provides power to the workpad 10 as well 
reference point. Each subsequently generated point is as display signals to operate the LCD 18 and also touch 
monitored with respect to a time period relative to the 55 signals to operate the overlay in both finger touch and 
time the reference point was generated. When subse- stylus modes. In addition, the cable 28 is also the con- 
quent point is generated at a time period greater than a duit to the computer for the measurement of the signal 
given time period, the delay timer is reset as in the strength received by the stylus 20 and of the frequency 
embodiment above. change due to changes in capacitance of a finger touch. 

BRIEF DESCRIPTION OF THE DRAWINGS 60 , FI °" \ sb ° T "? f ^ectural diagram of the finger 

touch and stylus detection system which is similar to 

The above objects, features and improvements will that disclosed in FIG. 9 of U.S. Pat. No. 4,686,332. The 

be better understood with reference to the following touch control processor 30, random access memory 32, 

drawing figures. read only memory and the I/O controller 36 are on a 

FIG. 1 shows the front view of an overlay unit used 65 touch panel adapter card 37 in a personal computer, 

for the detection of finger touch and stylus position. while the rest of the touch electronics are integrated in 

The unit is disposed over a flat panel display to form the touch workpad 10. As discussed in connection with 

what is known as a "touch workpad," FIG. 1, the touch workpad 10 communicates with the 
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personal computer and touch panel adapter card 37 Commonly assigned, co-pending application No. 
via-the workpad cable 28. The vertical X conductors 07/344,879, entitled "Advanced User Interface," filed 
and the horizontal Y conductors are connected through Apr. 28, 1989, which is hereby expressly incorporated 
the X bus 38 and the Y bus 40, respectively, to the wire herein by reference, describes an operating system ex- 
selection multiplexer 42. The radiative pickup stylus 20 5 tension which allows alternative forms of input to be 
is connected through the gate 44 to the radiative pickup handled by conventional software applications which 
measurement device 46. The wire selection multiplexer were not written to understand such alternative forms 
42 is connected through the mode multiplexer 50 to the of input. For example, conventional applications are 
capacitance measurement device 52 which is used for written to accept only keyboard and mouse input sig- 
capacitance ringer touch detection. The wire selection 10 nals. A user could use the Advanced User Interface 
multiplexer 42 is also connected through the mode mul- (AUI) to utilize a touch sensor without modifying any 
tiplexer 50 to the 40 kHz oscillator driver 54 which is of the software application code. In accordance with 
used to drive the X bus 38 and the Y bus 40 for the the present invention, AUI is stored in RAM 80 with 
stylus detection operation. The mode multiplexer 50 the operating system 81 and application programs 82. In 
also provides an enabling input to the gate 44 to selec- 15 a preferred embodiment, this invention improves AUI 
tively connect the output of the stylus 20 to the radia- by allowing it more efificientiy to distinguish between 
tive pickup measurement device 46 for stylus detection mouse commands and gesture or character data, 
operations. The output of the capacitance measurement FIG. 3 is a representation of the touch workpad of 
device 52 is connected through the analog-to-digital FIG. 1 showing an opened software application win- 
converter 56 to the workpad bus 58. The output of the 20 dow 100. The application is called "MAG.EXE." The 
radiative pickup measurement device 46 is connected work space area 102 is a typical feature of a window and 
through the analog-to-digital converter 48 to the bus 58. is the area where most data for input of the application 
The control input 60 of the wire selection multiplexer is generated. The action bar 104 contains a single select- 
42 is connected to the bus 58. The control input 62 of able menu item, "options." If the user touches down 
the mode multiplexer 50 also is connected to the bus 58. 25 with the pointing device (e.g., stylus, finger, etc.) on the 

The workpad bus 58 is connected via workpad inter- "options" menu item, a pulldown menu will appear to 
face 64 to the workpad cable 28 which is connected to allow him to select one of many available options, 
interface 66 in the touch panel adapter card 37 of the The window also depicts the inked trail of a set of 
personal computer (PC). The PC interface 66 communi- points representing a gesture 106 made by a pointing 
cates with the main system bus 68 and with the adapter 30 device. In this case, the gesture 106 is an "up arrow" 
card 70. The I/O controller 36 has an I/O bus 72 which which is one of many gestures which is recognizable by 
is connected to the main bus 68 of the PC. The I/O the AUI and which a user can use to cause the computer 
controller 36 is also connected to adapter card bus 70. system to take one of many possible actions. One possi- 
The adapter bus 70 also interconnects the control pro- ble meaning for the "up arrow" gesture, when input in 
cessor 30 with the read only memory (ROM) 34 and the 35 the work space area 102 of window 100, would be to 
random access memory (RAM) 32. The personal com- enlarge or to maximize the window 100. 
puter includes standard devices known in the art such as Also depicted in FIG. 3 is a long line 107 representing 
a CPU 74, ROM 76, disk storage 78, a memory 80 another trail of input points generated by a pointing 
which stores operating system 81 and application pro- device. For this series of points after touchdown, the 
grams 82, a standard keyboard 84, and a standard dis- 40 user moved, twelve points and then ceased further 
play 86. The standard display 86 is typically a cathode movement As shown in FIG. 3, the user did not move 
ray tube (CRT), and in the preferred embodiment the the pointing device from point 12 to point 35. Thus, a 
display 86 is in addition to the liquid crystal . display timeout was created and program execution transferred 
(LCD) 18 of workpad 10. to mouse emulation mode. Now an application com- 

The wire selection multiplexer 42 and the mode mul- 45 mand, such as a "file" command 108, could be activated 
tiplexer 50 connect selected patterns of a plurality of the by recognizing the series of points as a "file" command 
horizontal and vertical conductors in the overlay 16 to generated by a mouse-pointing device and sent to the 
either the capacitance measurement device 52 or the 40 application program. 

kHz oscillator driver 54, in response to control signals The present invention includes a computer module 
applied over the control inputs 60 and 62 from the bus 50 within the stylus or finger-based operating system ex- 
58 by the control processor 30. During finger touch tension (AUI) to differentiate between touch input sig- 
ope rations, the capacitance measuring device 52 has its nals intended to emulate a mouse command, such as a 
input coupled through the mode multiplexer 50 and the mouse button down, mouse move, or a mouse button 
wire selection multiplexer 42 to selected single conduc- up, and those touch input signals which are to be con- 
tors in the horizontal and vertical conductor array in 55 sidered a gesture or character. This differentiation is 
the overlay 16 in response to control signals from the done by using a time delay to differentiate between the 
control processor 30. The output of the capacitance two types of touch input signals. The AUI allows the 
measurement device 52 is converted to digital values by user to touch the screen and move to the desired posi- 
the A/D converter 56 and is supplied over the bus 58 to tion before the application is notified of the command 
the control processor 30. The control processor 30 60 by placing all the points received from the touch sensor 
executes a sequence of stored program instructions to in a stroke buffer. The points stored in the stroke buffer 
detect the horizontal array conductor pair and the verti- are hereinafter referred to as "a stroke." Once the user 
cal array conductor pair in the overlay 16 which are reaches the desired position and stops moving for the 
both being touched by the operator's finger. set time delay, a mouse command is generated at the 

The finger touch and stylus sensing modes operate 65 point at which the user has stopped. Since AUI recog- 
independently of one another, the detection system nizes the timeout as mouse emulation, rather than as a 
cycling between the two modes until a finger touch or gesture command, the stroke buffer is not used and the 
stylus is detected. application is passed a mouse command; in the preferred 
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embodiment the command would be a mouse button character, a mouse command would be generated at 

down message. For example, if the user stops moving 130. 

the pointing device at a desired position for 200 millisec- If the input stroke has been recognized by the gesture 

onds, a mouse command, such as a mouse down button recognition unit as a circle gesture, the AUI passes the 

command, at the desired position is communicated to 5 circle gesture signal to the appropriate computer mod- 

the application program. However, if the user starts to ule 132. 

move the device again before the end of the set period There is described below with reference to FIG. 5A 
of time, the command is not generated until the user and 5B, an operation of the invention to reduce system 
once again stops (without lifting off) and pauses for the overhead associated with resetting a delay timer which 
period of time delay. The time delay is typically defined 10 is used to determine whether a cessation of motion of 
in milliseconds (ms) and may be varied depending upon the pointing device has occurred, 
the screen area with which the user is interacting. The Referring to FIG. 5A, the user first touches a point- 
time delay may be specified by the user or may be al- ing device down on the touch sensor in 150 and gener- 
tered by the application program which is utilizing the ates one point In 152, after the first point is collected by 
touch input signal. 15 the AUI, the pointing device that generated that point is 
If, on the other hand, the user does not stop at a identified; that is, the pointing device is identified as a 
particular point for the specified time delay period and finger, stylus, mouse, etc. In 154, the point generation 
instead lifts off the touch screen, the AUI selects the set rate is determined. Each pointing device generates 
of input points generated by the pointing device (the points at different rates. The stylus, for example, gener- 
stroke) as candidate points for character or gesture 20 ates 1 10 points per second, whereas the mouse generates 
recognition. In other words, the time delay provides a 40 points per second. At 156, the system refers to a user 
window in time in which gestures can be made. During profile and retrieves a delay timer value. There is a 
this time, the AUI will only allow gestures to be made. different delay timer value for each pointing device. 
If the user waits for the time delay period before lifting For example, the stylus timer delay is 200 ms, whereas 
off, the points generated by the pointing device will not 25 the finger timer delay is 300 ms. In 158, the delay timer 
be candidates for a gesture. The stopping point coord!- value is divided by 2.3, an empirically derived constant, 
nates (i.e., coordinates corresponding to the last point to yield 86 ms (i.e., 200 ms divided by 2.3 yields 86 ms). 
generated by the pointing device prior to detection of At 160, the number of points expected during this 86 ms 
motion cessation) are then used for a mouse command. period is then calculated. With a stylus generating 1 10 
If the user lifts the pointing device before the time delay 30 points per second, there would be approximately 9 
period expires, an attempt will be made to recognize the input points during the 86 ms period. The system re- 
set of generated points as a gesture or character. If the cords the touch-down point and uses it as an anchor, 
points are not so recognized, however, the normal The term "anchor" implies that the next point gener- 
mouse emulation sequence will be generated. If the ated by the pointing device must be a predetermined 
points are recognized, the AUI sends the appropriate 35 distance from the touch-down point to be considered a 
commands to the appropriate application program. movement of the pointing device. The predetermined 
The recognition of a circle gesture made by a point- distance may be varied in accordance with the system 
ing device on the face of the touch overlay 16 is de- resolution. At 164, the point count is zeroed out. As the 
scribed below with reference to FIG. 4. points are received, they are counted. In 166, the delay 
A user, having determined that he wishes to invoke a 40 timer is started using the retrieved value. At this point, 
software utility by making a gesture with a pointing setup initialization is completed, 
device, draws a circle on the face of the touch sensor 16 At 188, the user lifts the pointing device off the touch 
and lifts the device off before the end of the set time sensor. If the user lifts off, there will be no mouse emu- 
delay. The touch sensor 16 generates a series of inter- lation detected and the system categorizes the input 
rupts to a device driver for the touch workpad at 110 45 points generated by the pointing device as a gesture or 
which passes a set of input signals to the AUI corre- handwriting input and cancels the timer. In 192, the 
sponding to the set of points in the circle at 112. delay timer times out The delay timer timed out be- 
When the AUI discovers the first touch input point cause the user stopped moving the pointing device for 
(i.e., a touch-down of the pointing device on the touch time in excess of the delay period and the system 
sensor 16), it looks up the time delay period selected by 50 stopped resetting the timer. When that happens, the 
the user or programmer at 120. At 122, the touch input system enters mouse emulation mode, 
points generated by the pointing device are collected in In FIG. 5B, at 168, each new point generated is col- 
a stroke buffer until the stylus lift-off event is detected. lected from the touch sensor. For the stylus, the points 
At 124, as the touch input points are received by the are received at 110 points per second. At 170, the point 
AUI. A delay timer is reset every nth pomt to determine 55 count is incremented. At 172, the X-Y coordinates of 
the period of stability or cessation of pointing device the newest input point are examined to see if the point is 
movement initiated by the user pausing on the touch more than two picture elements (pels) from the anchor 
sensor at 126. If the user moves the pointing device or the previous input point. If so, the system classifies 
before the time delay period has expired, the AUI con- the pointing device as having movement In 174, the 
tinues to receive the touch input points in the stroke 60 new point is recorded. The X-Y coordinates are used 
buffer at 122. If, however, the pointing device is not for the new anchor, and the timestarnp of that point is 
moved within a period that exceeds the time delays a saved. If there is no movement, the last two steps are 
mouse command is recognized and generated at 130. In omitted. At 178, the number of points received up to 
this example, the user does not pause before pointing this time is calculated. If the number is an integer multi- 
device (e.g., the stylus 20) is lifted, and therefore, the 65 pie of the divide rate, the system proceeds to 180 to see 
stroke is sent to a character recognition unit or gesture if motion had occurred within the last n points, ("n" in 
recognition unit for processing at 128: If the touch input this case is 9.) The divide rate for the stylus would be 9, 
stroke has not been recognized as either a gesture or so the system would check for multiples of 9 (e.g., 9, 18, 
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27, etc.)- If motion has occurred, the elapsed time is 
determined from the time the last motion point was 
received. If there has been any elapsed time since the 
last motion point, the elapsed time is subtracted from 
the original timer value, e.g., 200 ms. At 186, the timer 
delay is reset using the new timeout value which would 
be 200 ms minus the elapsed time between motion cessa- 
tion and the nth point In 178 and 180, if either of those 
tests resulted in no, the system exits. 

In FIG. 6, there is shown a stroke consisting of a 
touch-down and a series of nine (9) points. As discussed 
previously, after 9 points are received from the stylus, 
the system checks to see if movement has occurred 
within the last 9 points. In the figure, the stylus move- 
ment has been relatively rapid. In actual use, the points 
are likely to be spaced more closely together. 

In FIG. 6, there has been motion within the last 9 
points; the portion of the stroke from point 8 to point 9 
has been inked. After the 9th point is received, the delay 
timer, which is waiting for a period of stability of 200 20 
ms, is reset to zero and resumes waiting for the user to 
stop moving. 

In FIG. 7, the user has continued to move on to 
points 10, 11 and 12. Points 13-18 are shown in paren- 
thesis. After the user moved to point 12, he held the 25 
stylus stationary so that the last 6 points were generated 
in the same position as the 12th point At point 18, the 
system, again, determines if any motion occurred in the 
previous 9 points. Again, the answer is yes. Motion 
occurred on the first 3 points (10-12). However, on the 30 
last 6 points there was no motion, so the 200 ms timer is 
reset To account for the absence of motion for the last 
6 points, 54 ms, the timer is reset at the value of 200 ms 
minus 50 ms which equals 150 ms. 

FIG. 8 shows that the user never did again move the 35 
stylus. The stylus remained stationary after point 12, 
and points 19-35 show there was no movement so the 
timer is never reset At point 35, a timeout occurred 
because the user failed to move the pointing device for 
200 ms. When the system detects a timeout, mouse 40 
emulation mode is entered to cause a mouse command 
input message to be produced. 

The AUI system provides facilities for finger- and 
stylus-based user interface devices. When used with 
devices capable of tracking the movement of a user's 45 
finger or the movement of a stylus, AUI provides sup- 
port for the handling of the resulting stroke informa- 
tion. These facilities include the visual tracing on the 
screen of ink flowing from the stylus, the delivery of the 
stroke information to application programs, and the 50 
delivery of the stroke information to recognition sub- 
systems for further analysis. 

AUI employs a user interface technique where mouse 
emulation for the pointing device (stylus, finger or 
mouse) is delayed until the user stops moving the point- 55 
ing device. Any stroke completed (i.e., pointing device 
is lifted off touch sensor) before movement has stopped 
is classified as a gesture or handwriting input. After 
motion stops, it is assumed the user intends to emulate a 
mouse input, and a mouse emulation mode begins. To 60 
effect the detection of motion cessation, the user must 
touch down and hold the pointing device stationary for 
a timeout period. "Stationary" is defined as no change 
in position greater than n pels, where 4 n pels' can be 
expressed in either display or sensor resolution units. In 65 
one preferred embodiment, a value of two display pels 
is used. Examples of typical timeout values are 200 ms 
for a stylus, 300 ms for finger touch, and 200 ms for a 
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mouse. These values represent a trade-off between mini- 
mizing the delay to enter mouse emulation and making 
gestures easy to stroke without inadvertently timing out 
during the stroke. 

The invention has particular application in a multi- 
tasking computer system. As described above, a single 
task or thread is typically dedicated to managing a delay 
timer used to determine motion cessation. In the prior 
art, for example, a stylus pointing device generates 110 
points per second, one point every 9 ms. Thus, the prior 
art processor would need to switch to and from the 
timer thread every 9 ms when the stylus is in motion. As 
a result, 100% of the processing time of the system is 
taken up by managing the timer, assuming the processor 
takes 9 ms to process data for each point generated. 

In accordance with the present invention, this timer 
thread needs to be reset only every nth point generated 
by the pointing device used with the computer system. 
Thus, the system processor does not have to waste 
processing time to switch between a currently execut- 
ing thread to the timer thread after each point is gener- 
ated by a moving pointing device. 

To clarify the above description invention, the fol- 
lowing pseudocode is provided to detail the logic flow: 



when a user touches down 
identify stroke device 
determine point rate 

retrieve the delay timer value from user profile 

divide delay timer value by 2.3 

calculate number of points expected during this 

period 
clear point counter 
start delay timer 
endwhen 

when a new point is received from the sensor 

increment number of points received in the stroke 
if point shows movement greater than 2 pels from 

last motion point, record point and 

times tamp of point 
endif 

if number of stroke points is even multiple of 
time divide rate 

if motion had occurred during last n points 
determine elapsed time since last 

motion point received 
subtract this time from original delay 

timer value 
reset the delay timer using this new 

timeout value 

endif 
endif 
endwhen 

when user lifts off 

cancel delay timer 
endwhen 

when delay timeout occurs (asynchronously) 

enter mouse emulation mode 
endwhen 



While the invention has been particularly shown and 
described with reference to a preferred embodiment 
thereof, it will be understood by those skilled in the art 
that various changes in detail may be made therein 
without departing from the spirit, scope and teaching of 
the invention. For example, the above embodiment 
assumed a steady generation of points was made by the 
pointing device such that the timer used to determine 
cessation of a moving pointing device could be periodi- 
cally reset after n points were generated. However, in a 
system where the movement of the pointing device is 
sporadic and a steady stream of points are not generated 



by a moving pointing device, the timer can be reset 
periodically at timed intervals, e.g M 86 ms, in accor- 
dance with the following alternative embodiment of the 
invention. 

In this embodiment, an initial point generated by the 5 
pointing device is used as an anchor point (as in the 
embodiment described above). All subsequent points 
are then monitored as to their time period of generation 
(hereinafter referred to as a "timestamp") relative to the 
anchor point was generated. When a point is generated 10 
with a timestamp greater than a given interval of time, 
e.g., 86 ms, the timer is reset and the point is used as the 
new anchor point. According to this embodiment of the 
invention, the timer is periodically reset at given time 
intervals when points are generated within the predeter- 15 
mined time-delay period, e.g., 200 ms. If no points are 
generated by the pointing device, and no movement 
thereof is detected, then a timeout occurs and a the 
system enters a mouse emulation mode (as described 
above). 20 

The embodiments presented above are for purposes 
of example and illustration only and are not to be taken 
to limit the scope of the invention narrower than the 
scope of the appended claims. 

Having thus described my invention, what I claim as 25 
new, and desire to secure by Letters Patent is: 

1. A method for reducing the processing time re- 
quired to recognize the cessation of motion of a moving 
pointing device in an information processing system, ~ 
the method comprising the steps of: 
determining that a pointing device is located in opera- 
tive proximity to a display screen, in said informa- 
tion processing system; 
generating a series of delay timers having a predeter- 35 
mined rate, while said pointing device is located in 
said operative proximity to said display screen, in 
said information processing system; 
accessing a predetermined timer divide rate value 4 n' 
from said information processing system, repre- ^ 
senting a predetermined plurality of *n' of coordi- 
nate point signals, in said information processing 
system; 

receiving a first point signal for said pointing device 
with respect to said display screen, and buffering 45 
said first point signal in a memory as an anchor 
point and starting a delay timer, in said information 
processing system; 

receiving a plurality of n — 1 succeeding point signals 
for said pointing device with respect to said display 50 
screen, and buffering said plurality of n -J- 1 signals, 
in a memory in said information processing system; 

computing whether said pointing device has moved 
with respect to said display screen by comparing at 
least one of said buffered plurality of n— 1 point 55 
signals with said buffered anchor point, in said 
information processing system; 

if said computing step determines that said pointing 
device has moved, then receiving a second plural- 
ity of n succeeding coordinate point signals for said 60 
pointing device with respect to said display screen, 
and buffering a first of said second plurality of n 
signals as a new anchor point while restarting the 
delay timer, in said information processing system; 

if said computing step determines that said pointing 65 
device has not moved, then outputting a first out- 
put signal indicating cessation of motion of said 
pointing device; 
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if said determining step determines that said pointing 
device is not located in said operative proximity to 
said display screen, then outputting a second out- 
put signal indicating that said pointing device has 
been removed from said operative proximity to 
said display screen. 

2. The method as recited in claim 1, which further 
comprises the steps of: 

identifying the pointing device as one among several 

possible pointing devices; and 
setting, in accordance with the identified pointing 

device, said predetermined delay timer value. 

3. The method as recited in claim 1, wherein the 
pointing device is a finger moving across a touch sen- 
sor, and further comprising the step of canceling said 
delay timer when the finger is lifted off the touch sen- 
sor. 

4. The method as recited in claim 1, wherein the 
information processing system enters a mouse emula- 
tion mode in response to said first output signal. 

5. The method as recited in claim 1, further compris- 
ing the step of entering a gesture recognition mode 
when said second output signal is outputted. 

6. A method for reducing the processing time re- 
quired to recognize the cessation of motion of a moving 
pointing device in an information processing system, 
the method comprising the steps of: 

determining that a pointing device is located in opera- 
tive proximity to a touch sensor controlling a dis- 
play screen, in said information processing system; 

generating a series of delay timers having a predeter- 
mined rate, while said pointing device is located in 
said operative proximity to said touch sensor, in 
said information processing system; 

accessing a predetermined timer divide rate value V 
from said information processing system, repre- 
senting a predetermined plurality of *n' of coordi- 
nate point signals, in said information processing 
system; 

receiving a first point signal for said pointing device 
with respect to said touch sensor, and buffering 
said first point signal in a memory as an anchor 
point and starting a delay timer, in said information 
processing system; 

receiving a plurality of n— 1 succeeding coordinate 
point signals for said pointing device with respect 
to said touch sensor, 

and buffering said plurality of n— 1 signals, in said 
memory in said information processing system; 

computing whether said pointing device has moved 
with respect to said touch sensor by comparing at 
least one of said buffered plurality of n— 1 point 
signals with said buffered anchor point, in said 
information processing system; 

if said computing step determines that said pointing 
device has moved, then receiving a second plural- 
ity of n succeeding coordinate point signals for said 
pointing device with respect to said touch sensor, 
and buffering a first of said second plurality of n 
signals as a new anchor point while restarting the 
delay timer, in said information processing system; 

if said computing step determines that said pointing 
device has not moved, then outputting a first out- 
put signal indicating cessation of motion of said 
pointing device; if said determining step determines 
that said pointing device is not located in said oper- 
ative proximity to said touch sensor, then output- 
ting a second output signal indicating that said 



pointing device has been removed from said opera- 
tive proximity to said touch sensor. 

7. The method as recited in claim 6, further compris- 
ing the step of identifying a type of said pointing device 
wherein said predetermined delay timer value is deter- 5 
mined in accordance with the type identified. 

8. The method recited in claim 6, wherein when mo- 
tion of said pointing device has not been detected within 
a number of previous position points, said predeter- 
mined delay timer value is reduced in accordance with 10 
the number of previous position points. 

9. The method recited in claim 6, wherein the system 
accepts information from said pointing device as ges- 
ture or character information in response to movement 
and removal of said pointing device from said touch 15 
sensor. 

10. The method as recited in claim 6, further compris- 
ing the step of entering a gesture recognition mode 
when said second output signal is outputted. 

11. A method for reducing the processing time re- 20 
quired in a multi-tasking information processing system, 

to recognize the cessation of motion of a moving first 
pointing device that emulates a second pointing device 
in the system, the method comprising the steps of: 
determining that a first pointing device is located in 25 
operative proximity to a display screen, in said 
information processing system; 
generating a series of delay timers having a predeter- 
mined rate, while said pointing device is located in 
said operative proximity to said display screen, in 30 
said information processing system; 
accessing a predetermined timer divide rate value *n* 
from said information processing system, repre- 
senting a predetermined plurality of V of coordi- 
nate point signals, in said information processing 35 
system; 

receiving a first point signal for said pointing device 
with respect to said display screen, and buffering 
said first point signal in a memory as an anchor 
point and starting a delay timer, in said information 40 
processing system; 

receiving a plurality of n— 1 succeeding point signals 
for said pointing device with respect to said display 
screen, and buffering said plurality of n — 1 signals, 
in said memory in said information processing sys- 45 
tern; 

computing whether said pointing device has moved 
with respect to said display screen by comparing at 
least one of said buffered plurality of n— 1 point 
signals with said buffered anchor point, in said 50 
information processing system; 

if said computing step determines that said pointing 
device has moved, then receiving a second plural- 
ity of n succeeding coordinate point signals for said 
pointing device with respect to said display screen, 55 
and buffering a first of said second plurality of n 
signals as a new anchor point while restarting the 
delay timer, in said information processing system; 

if said computing step determines that said first point- 
ing device has not moved, then outputting a first 60 
output signal indicating cessation of motion of said 
first pointing device and designating it as said sec- 
ond pointing device; 

if said determining step determines that said first 
pointing device is not located in said operative 65 
proximity to said display screen, then outputting a 
second output signal indicating that said first point- 
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ing device has been removed from said operative 
proximity to said display screen. 

12. A data processing system for reducing the pro- 
cessing time required to recognize the cessation of mo- 
tion of a moving pointing device, comprising: 

means for determining that a pointing device is lo- 
cated in operative proximity to a touch sensor con- 
trolling a display screen, in said data processing 
system; 

means for generating a series of delay timers having a 
predetermined rate, while said pointing device is 
located in said operative proximity to said touch 
sensor, in said data processing system; 

means for accessing a predetermined timer divide rate 
value V from said information processing system, 
representing a predetermined plurality of V of 
coordinate point signals, in said data processing 
system; 

means for receiving a first point signal for said point- 
ing device with respect to said touch sensor, and 
buffering said first point signal in a memory as an 
anchor point and starting a delay timer, in said data 
processing system; 

said receiving means receiving a plurality of n— 1 
succeeding point signals for said pointing device 
with respect to said touch sensor, and buffering 
said plurality of n— 1 signals, in a memory in said 
data processing system; 

means for computing whether said pointing device 
has moved with respect to said touch sensor by 
comparing at least one of said buffered plurality of 
n — 1 point signals with said buffered anchor point, 
in said data processing system; 

if said computing step determines that said pointing 
device has moved, then receiving a second plural- 
ity of n succeeding coordinate point signals for said 
pointing device with respect to said touch sensor, 
and buffering a first of said second plurality of n 
signals as a new anchor point while restarting the 
delay timer, in said data processing system; 

if said computing means determines that said pointing 
device has not moved, then said computing means 
outputs a first output signal indicating cessation of 
motion of said pointing device; 

if said determining means determines that said point- 
ing device is not located in said operative proxim- 
ity to said touch sensor, then said computing means 
outputs a second output signal indicating that said 
pointing device has been removed from said opera- 
tive proximity to said touch sensor. 

13. The data processing system recited in claim 12, 
wherein the system accepts information from said point- 
ing device as mouse input data in response to a recogni- 
tion of motion cessation. 

14. The data processing system recited in claim 12, 
wherein said touch sensor is a touch-sensitive display 
screen that displays information under control of said 
pointing device. 

15. The data processing system recited in claim 12, 
wherein the system is a multi-tasking computer system 
that dedicates a single task to managing said computing 
by said computing means. 

16. The data processing system recited in claim 12, 
wherein the system enters a gesture recognition mode 
when said second output signal is outputted. 

***** 



